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Abstract 


The  operational  performance  of  electro-optical  imaging  systems  is  currently  predicted 
using  models  based  on  an  approach  introduced  by  Johnson  in  1958.  This  model  has  been 
shown  to  be  inadequate  when  used  to  predict  performance  over  a  wide  range  of  conditions 
of  signal-to-noise  ratio  and  range  to  the  target. 

This  paper  describes  our  efforts  to  develop  and  validate  a  model  which  is  more 
effective  than  the  Johnson  lodel  in  predicting  image  quality.  Implications  in  performance 
prediction  modeling  and  electro-optic  sensor  design  tradeoffs  between  sensitivity  and 
resolution  are  also  discussed. 


Introduction 


Over  the  past  twenty  years  we  have  seen  many  changes  in  electro-optical  imaging 
systems-  Evolutionary  changes  in  photocathodes,  television  caunera  tubes,  and  infrared 
detectors  have  occurred.  We  have  also  seen  revolutionary  changes  such  as  microchannel 
plate  intensifiers ,  charge  coupled  devices,  and  infrared  focal  plane  arrays.  During 
this  same  period  the  ** Johnson  Criterion"  has  endured  as  the  foundation  for  models  pre¬ 
dicting  field  performance  of  military  imaging  systems.  However,  laboratory  and  field 
experiments  done  in  1974  by  George  O'Neill^  of  the  Naval  Air  Development  Center  question 
the  accuracy  of  the  Johnson  model  when  applied  over  a  wide  range  of  video  signal-to-noise 
ratios  (SNR) .  Over  the  past  several  years,  as  our  involvement  with  "next  generation" 
infrared  imagers  has  grown,  we  have  become  increasingly  aware  of  the  need  for  a  model 
that  better  describes  the  quality  of  electronically  generated  images.  This  paper 
describes  the  progress  of  our  effort  over  the  past  year  to  develop  such  a  model. 

The  Johnson  criterion 

2 

In  1958  Johnson  described  an  approach  for  estimating  the  maximum  range  at  which  a 
given  task,  e.g.,  detection,  recognition,  or  identification  of  a  target,  can  be  done  with 
an  imaging  system.  This  approach  is  based  on  the  limiting  resolution  of  that  system  as 
measured  under  the  same  ambient  conditions  encountered  in  the  field.  According  to  the 
Johnson  concept,  if  at  any  given  range  the  minimum  target  dimension  subtends  a  specified 
number  of  cycles  of  a  periodic  pattern  resolvable  under  the  conditions  of  the  test,  that 
target  could  be  discriminated  at  the  corresponding  level.  The  process  of  relating  a  test 
pattern  to  the  target  of  interest  is  illustrated  in  figure  1.  The  specific  number  of 
resolvadsle  half-cycles  across  the  minimum  dimension  required  to  detect,  recognize,  or 
identify  was  determined  empirically  from  field  experiments  involving  such  instruments  as 
an  optical  telescope,  image  intensifiers,  and  an  intensified  image  orthicon  television 
camera,  with  which  targets  such  as  soldiers,  artillery  pieces,  and  army  vehicles  were 
viewed.  It  is  reasonfUsle  to  assume  that  these  decision  levels  might  be  different  for  other 
types  of  targets  and  backgrounds.  For  example,  the  number  of  half-cycles  required  to 
positively  identify  a  warship  may  be  quite  different  than  Johnson's  value  of  12.8.  Also, 
it  may  be  more  appropriate  to  use  a  two-dimensional  criterion  to  accommodate  changes  in 
viewing  aspect.  Moser suggested  generalizing  Johnson's  criterion  by  specifying  the 
number  of  square  picture  elements  or  pixels  that  the  target  area  must  subtend  before  a 
given  task  can  be  accomplished.  The  pixel  size  is  defined  by  the  width  of  a  half-cycle 
of  a  bar  pattern  resolvable  under  the  conditions  being  considered. 

Beyond  the  question  of  the  exact  number  of  half-cycles  or  pixels  needed  to  do  a 
given  task  there  is  a  more  fundamental  question:  does  image  quality  or  information 
content  correlate  well  with  limiting  resolution?  Implicit  in  Johnson-based  models  is  the 
assumption  that  two  images  of  a  given  target  are  of  equal  utility  if  the  number  of  reso¬ 
lution  cells  subtended  by  the  target  is  equal.  That  this  is  sometimes  not  the  case  is 
illustrated  in  figure  2.  The  four  images  in  this  figure  were  photographed  from  the  cathode 
ray  tube  (CRT)  display  of  a  low  light  level  television  (LLLTV)  system  using  an  inten- 
sified-silicon  intensified  target  (I-SIT)  tube  chain.  *^6  two  upper  pictures  are  of  a 
standard  test  chart  used  to  measure  limiting  resolution.  In  the  left-hand  image,  the 
pattern  brightness  is  adjusted  to  yield  the  highest  acuity  achievable  with  this  system  in 
the  horizontal  direction:  525  *25  lines  or  half-cycles  per  picture  height.  The  right- 
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hand  image  is  produced  at  a  pattern  brightness  reduced  to  the  point  whore  horizontal 
resolution  is  shot-noise-limited  to  200  +25  lines  per  picture  height.  The  lower  pair  of 
pictures  was  made  using  ship  silhouettes  of  the  same  highlight  brightness  levels  as  in  the 
pictures  above.  The  scale  of  the  silhouettes  is  adjusted  so  that  the  size  of  a  resolution 
cell  at  the  simulated  target  range  is  nearly  the  same  at  both  brightness  levels:  3.6  feet 
in  the  left-hand  image;  3.1  feet  in  the  right-hand  image.  According  to  the  Johnson  cri¬ 
terion,  the  right-hand  image  should  be  slightly  better  since  more  bars  at  the  limiting 
resolution  are  subtended  by  the  nominal  ship  height.  Similarly,  since  both  images  are  of 
the  same  broadside  view  of  the  ship,  Moser's  pixel  criterion  would  favor  the  right-hand 
image  slightly.  Under  these  same  conditions,  NAVAIRDEVCEN  observers  viewing  the  monitor 
and  the  original  photographs  agree  that  the  left-hand  image  provides  much  more  information 
about  the  ship. 


Fig  1.  The  Johnson  criterion  relates  limiting  resolution  density 
to  a  critical  target  dimension 

Broadband  quality  measures 

There  is  a  reasonable  explanation  for  the  apparent  failure  of  the  Johnson  model  to 
determine  image  quality  over  the  range  of  conditions  spanned  by  figure  2.  In  the  left-hand 
exposure,  SNR  is  high,  and  acuity  is  limited  by  the  spatial  frequency  response  of  the 
various  elements  in  the  image  producing  chain.  Under  these  conditions  a  series  of  bars 
blur  together,  so  that  the  observer  cannot  distinguish  or  resolve  them.  However,  most  of 
the  detail  useful  in  distinguishing  ship  silhouettes  is  aperiodic  in  nature.  Since  the 
signal  power  contained  in  images  of  aperiodic  detail  such  as  spars  or  superstructure  is 
spread  across  the  spatial  frequency  spectrum,  it  does  not  suffer  as  greatly  as  the  periodic 
bar  patterns  when  range  is  increased.  In  the  right-hand  exposure  it  is  SNR  rather  than 
spatial  frequency  response  that  limits  resolution.  Since  the  noise  is  broadband,  it 
limits  aperiodic  signals  at  least  as  much  as  periodic  signals. 

This  heuristic  explanation  suggests  that  a  model  which  weighs  signal  and  noise  over 
the  entire  video  spectrum  might  be  more  effective  in  determining  image  quality,  at  least 
for  ship  targets,  where  aperiodic  detail  is  prevalent.  This  concept. of  a  broadband 
frequency-weighted  summary  measure  is,  of  course,  not  new.  Biberman^  cites  the  work  of 
Hufnagel  in  1965,  which^offers  several  broadband  measures  for  assessing  the  image  quality 
of  photographs.  Snyder  extends  to  electronically  produced  images  a  similar  concept 
introduced  by  Charman  and  Olin  in  1965,  and  demonstrates  correlation  between  the  resulting 
measure  and  observer  performance  with  raster-scan  images.  The  focus  of  our  work  over  the 
past  year  was  to  develop  these  and  other  broadband  quality  measures,  and  to  compare  their 
effectiveness  in  predicting  relative  image  quality  to  each  other  and  to  the  Johnson  model. 

Approach 

Briefly,  our  approach  for  establishing  a  model  that  adequately  predicts  relative 
quality  was  to  generate  a  series  of  images  over  a  wide  range  of  video  SNR,  sensor  mod¬ 
ulation  transfer  function  (MTF) ,  and  target  scale.  The  images  were  generated  by  photo¬ 
graphing  the  display  of  a  vidicon  camera  system  viewing  silhouettes  of  the  broadside  view 
of  a  Soviet  KOTLIN  class  destroyer.  Observers  were  then  asked  to  rank  order  the  images 
according  to  the  amount  of  information  they  could  glean  from  each  exposure.  This  rank 
ordering  was  then  correlated  to  the  orderings  dictated  by  several  candidate  figures  of 
merit,  including  the  Johnson  criterion.  The  degree  of  correlation  and  the  lack  of  many 
obvious  mismatches  or  "wild  points"  in  the  correlation  plot  provides  an  indication  of  the 
effectiveness  of  a  given  figure  of  merit.  Following  is  a  more  detailed  account  of  the 
image  generation  process  and  of  the  technique  for  establishing  observer  ranking. 
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rig  Z,  Comparison  of  video  imagery  at  high  (ieftj  and 
low  (right)  s  ignal- to-no ist*  ratio 


Image  generation 


Figure  J  is  a  block  diagram  of  the  facility  used  to  generate  the  scries  ot  h 
images.  Transparencies  of  the  broadside  KOTLIN  silhouette  were  made  with  the  sea 
ranging  from  13b  to  1810  meters  per  picture  height.  These  transparencies  were  ba 
minated  in  a  b  10-mm  focal  length  collimator,  with  a  wc  1 1  ~  rcau  la  ted  incanvlescen  t  1 
intimony  tri sulfide  vidicon  camera  with  a  75  mm  focal  length  lens  was  alianed  wit 
collimator  to  produce  the  electronic  image  of  the  ship.  T!ie  3C3~'i  ino,  30-frame-'i 
video  signal  and  white  noise  were  addeci  and  passed  through  an  S-MHz  low  pass  filt*' 
Signal  and  noise  levels  were  controlled  independently  to  achieve  the  desired  SNR, 
the  same  time  filling  the  brightness  range  of  the  display  without  ciippinc  •):  the 
This  condition  was  insured  through  an  elaborate  procedure  usinq  an  e iect roni caj ly 
ated  gray  scale  and  a  photometer  to  measure  the  resultant  vlisplay  brightness.  Vi 
signal  was  determined  by  digitally  measuring  the  mean  dirference  between  the  volt 
points  within  the  silhouette  and  the  voltage  in  the  darker  backaround.  The  root- 
square  (RMS)  noise  was  determined  by  measuring  the  f rame- t o- t rame  variance  of  tht' 
at  a  point  in  or  around  the  silhouette.  SNR  is  defint'i  lun'em  as  the  ratio  of  tn 
ference  signal  to  the  RMS  noise. 
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The  spatial  frequency  response,  e.xpressed  in  terms  ct  the  MTF  was  contr'^llcvi  b\ 
defocussing  the  television  camera  lens.  MTF  was  measured  ^sina  the  same  min i com::ut cr- 
based  system  used  to  determine  signal  and  noise.  Schade  '  s  ‘  no  ise-oguiva  lent.  t>assband,  d  ^ , 
was  used  to  describe  MTF  with  a  single  number.  N  is  defined  as: 


JO 


N  \  (MTF  (f)  )  ^  df .  (1) 

®  )o 


For  this  experiment  two  values  of  were  used:  45  and  185  lines  per  picture  height. 

Each  image  was  photographed  from  a  21-inch  CRT  with  a  35  mm  single  lens  reflex  camera. 
Kodak  Plus-X  film  was  used  for  the  black  and  white  prints;  Ektachrome  200  with  color - 
correcting  filters  was  used  to  make  35  mm  slides.  Trial  exposures  showed  that  the  light 
and  dark  banding  produced  by  the  temporal  and  spatial  relationship  between  the  camera 
focal  plane  shutter  and  the  TV  raster  scanning  would  be  objectionable.  This  banding  was 
avoided  by  blanking  the  video  so  that  while  the  shutter  was  open  the  monitor  displayed 
exactly  three  frames  of  video.  The  0.1-second  exposure  time  was  chosen  to  simulate  the 
noise  equivalent  integration  time  of  the  typical  human  observer  viewing  a  display  of  about 
30  footlamberts . 

Eight  values  of  SNR,  6  scale  factors,  and  2  N  factors  were  used  to  make  up  the  total 
of  96  different  images  generated.  These  values  wef e :  0,3,  0.5,  1.0,  2.0,  4.0,  8.0,  12.0, 
and  100.0  for  SNR;  136,  197,  388,  543,  724,  and  1810  meters  per  picture  height  for  the 
scale  factor;  and  45  and  185  TV  lines  per  picture  height  for  N  .  Figure  4  is  a  composite 
of  the  48  images  produced  for  N^  =  185. 
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Fig  3.  Block  diagram  of  the  equipment  used  to  generate  test  Images 


Image  ranking 

A  group  of  engineers  and  physicists  was  used  to  establish  subjective  rank  ordering  of 
the  96  images.  All  had  vision  corrected  to  20/20;  none  was  specifically  trained  in  ship 
identification.  Prior  to  the  rank  ordering  process,  each  observer  was  shown  a  print  of 
the  highest  quality  image  of  the  ship,  and  was  instructed  to  note  the  position  and  appear¬ 
ance  of  guns,  masts,  and  superstructure.  He/she  was  then  shown  pairs  of  images  projected 
by  two  slide  projectors,  with  brightness  carefully  matched  and  maintained.  The  distance 
between  the  observers  and  the  screen  was  maintained  at  twice  the  height  of  the  projected 
slide.  He/she  was  given  unlimited  time  to  decide  which  of  the  two  slides  conveyed  more 
information  about  the  ship.  Sequencing  of  the  pairs  was  governed  by  the  observer's 
response,  using  a  procedure  that  avoided  showing  pairs  with  obviously  disparate  image 
quality.  In  this  manner,  all  significant  comparisons  were  completed  in  less  than  3-hours 
of  total  viewing  time,  broken  up  into  three  sessions  on  2  days,  with  15-minute  breaks  after 
20  minutes  of  viewing.  At  the  end  of  the  procedure  the  observer  was  shown  the  slides  in 
the  order  that  his/her  responses  dictated,  and  was  given  the  opportunity  to  reverse  the 
decision  on  close  calls.  Orderings  were  determined  for  nine  different  people.  Ranking 
was  also  done  by  several  observers  using  5-in.  x  7-in.  black  and  white  prints.  These 
orderings  proved  consistent  with  those  done  using  slides. 
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Results 


The  rankings  determined  by  the  observers  were  correlated  with  the  rankings  determined 
by  13  different  figures  of  merit  formulations.  The  formulations  that  produced  the  best 
correlations  are  described  below; 


Figures  of  merit 

Following  the  work  of  Snyder,  we  evaluated  an  image  quality  model  termed  Modulation 
Transfer  Function  Area  defined  as  the  area  between  the  MTF(f)  curve  and  the  observer 
demand  function,  D(f).  This  model,  referred  to  as  has  the  form; 

(MTF(f)  -  D(f)  )  df. 

Where  f.  is  the  intersection  of  the  two  functions  as  shown  in  figure  5. 


Figure  5,  Parameters  used  for 
image  quality  models 


The  demand  function  is  defined  as  the  mod¬ 
ulation  required  to  resolve  a  periodic 
pattern  at  a  given  frequency,  f,  given  the 
video  SNB.  Using  the  results  of  Col tman  and 
Anderson'  it  can  be  shown  that  over  a  broad 
range  of  SNR 

D(f)  ^  f.  (K  •  SNR)  ,  (3) 

where  K  is  a  constant  dependent  on  system 
parameters-  For  our  S-MHz  bandwidth  and  4:3 
picture  length  to  width  ratio. 

K  =  1000  lines  per  picture 

height  (4) 

As  stated  in  equation  (1) ,  Q.  would  have 
the  units  of  lines  per  picture  height.  This 
is  referred  to  target  space  by  dividing  by  the 
scale  factor,  H,  with  units  of  meters  per 
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picture  height; 


Q  =  Qi/H  (lines/meter)  (5) 

1  ^ 

A  second  model  that  was  evaluated,  termed  Modulation  Transfer  Function  Power  / 
is  based  on  the  integrated  signal  power  divided  by  the  integrated  noise  power: 


Q2  = 


i" 


S^(f)  (MTF)^(f)  df 


N^(f)  df 


(6) 


Assuming  both  the  signal  and  noise  spectra  are  uniform,  (5)  can  be  reduced  to 

the  form 

Q,  =  SNR^  N  (MTF)^  (f)  df.  (7) 

/o 

Substituting  equation  (1)  into  (7)  and  normalizing  by  the  scale  factor  yields: 

q'  =  (SNR) ^  (8) 

2  H 


,  A  third  model,  termed  Signal- to- Noise  Ratio  Threshold  (SNRT) ,  referred  to  as 

Q  is  a  variant  of  the  last: 

3'  • 

Q  =  SNR  (9) 

3  SNR  +  SNR  H 

o 

where  SNR  is  a  threshold  value,  empirically  determined  to  yield  the  highest  correlation 
rankings.^  This  figure  of  merit  herein  will  be  referred  to  as  SNRT. 

In  addition  to  these  models  and  many  variations  thereof,  the  Johnson-based  formulation 
was  also  evaluated.  Since  according  to  Johnson,  limiting  resolution  at  the  target  plane 
is  sufficient  to  define  quality,  this  model  termed  Q  is  simply  expressed: 

4 

(10) 

In  our  investigations  limiting  resolution,  f^,  was  calculated  using  the  demand  function 
and  MTF  as  illustrated  in  figure  5. 

Correlation  of  rankings 

The  effectiveness  of  a  given  model  in  predicting  image  quality  can  be  gauged  by 
comparing  the  rank  ordering  as  determined  with  the  model  and  the  ordering  determined  by 
observation.  Two  approaches  for  comparing  these  orderings  were  used.  The  first  was  to 
plot  the  figure  of  merit  calculated  for  the  image  placed  by  an  observer  in  a  given  posi¬ 
tion  against  the  figure  of  merit  of  the  image  that  would  have  been  in  that  position  if  the 
model  were  correct.  For  example,  if  the  picture  that  was  tenth  best  in  an  observer's 
estimation  had  a  calculated  figure  of  merit  of,  say  A,  but  the  picture  judged  by  the  model 
to  be  tenth  best  had  a  figure  of  merit  equal  to  B,  this  data  point  would  be  represented  by 
(A,  B) .  In  this  manner,  the  cross  correlation  of  a  given  observer's  responses  to  a  given 
model's  prediction  was  determined  for  all  of  the  observers,  and  for  many  variations  of  the 
models  previously  described. 

The  correlation  coefficient  proved  to  be  a  very  insensitive  measure  of  agreement.  In 
no  case  was  the  coefficient  less  than  0.9.  The  Johnson  criterion  yielded  values  ranging 
from  0.943  to  0.973,  depending  on  the  observer.  Figure  6a  is  the  cross  plot  of  the 
Johnson  ranking  with  that  of  a  typical  observer.  The  SNRT  model  defined  by  equation  (9) 
with  the  threshold  constant  set  equal  to  3.0  yielded  the  highest  coefficients,,  ranging 
from  0.987  to  0.993.  Figure  6b  is  the  plot  of  this  model  against  the  responses  of  the 
same  observer  as  in  6a.  These  same  correlations  were  also  done  on  a  logarithmic  scale,  so 
that  a  change  in  quality  at  the  low  end  of  the  scale  is  given  equal  weight  as  a  pro¬ 
portional  change  at  the  high  end.  Coefficients  were  typically  5%  lower  than  those  cal¬ 
culated  on  the  linear  scale.  Figures  7a  and  7b  are  the  logarithmic  plots  of  the  results 
with  the  Johnson  and  SNRT  models  corresponding  to  those  in  figure  6, 

The  second  approach  used  to  evaluate  the  effectiveness  of  a  model  in  predicting 
observer  response  was  to  consider  the  number  of  points  which  fall  far  from  the  prediction 
line  in  any  given  model.  Comparing  figure  6a  with  6b,  and  7a  with  7b,  it  appears  that 
despite  the  relatively  high  correlation  coef  f.'*  cient,  the  Johnson  model  is  capable  of 
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inalcing  much  larger  errors  in  estimating  image  quality.  In  table  I,  12  notable  “wild 
points"  are  analyzed.  For  each  of  the  12  points,  the  rank  assigned  to  our  "typical" 
observer  is  compared  with  that  determined  from  the  Johnson  and  SNRT  models.  Also  given 
are  the  corresponding  sensor  parameters.  Note  that  the  Johnson  model  tends  to  under¬ 
estimate  the  quality  of  images  with  high  SNR  values  while  overestimating  the  quality  of 
those  with  low  SNR.  SNRT,  while  not  perfect,  yields  much  better  agreement,  and  does  not 
consistently  favor  high  or  low  SNR  images. 


COMIUnON  Of  OKOVn  vs  jonnson 


COtRIUTION  Of  OBOrVIK  VS  $1% 


IMA€t  QUALITY  AS  ASSiSSED  lY  OISEIVEI 

6a  6b 

Correlation  of  image  quality  as  assessed  by  observers  and  predicted  by  model 

Linear  scale 


coM&ATioM  Of  omnra  vs  jqnnmm 


COMEBAYION  Of  OMtVa  VS  SNtr 


MMt  OUUITT  U  tSSBSn  IT  OHIITII 

7a 

Correlation  of 

image  quality  as  assessed  by  observer 
Log  scale 

7b 

and  predicted  by  models 

Table  I. 

Analysis  of  "Wild  Points" 

Rankings 

Parameters 

Observer 

Johnson 

SNRT  SNR  Scale (M/PH) 

'’e(L/PH) 

24 

3 

25 

1 

40 

1 

46 

2 

49 

6 

50 

1 

64 

3 

74 

9 

76 

9 

79 

5 

84 

5 

86 

6 

6 

4.0 

543 

4 

1.0 

197 

9 

1.0 

136 

5 

0.5 

197 

4 

12.0 

1810 

9 

0.3 

136 

0 

0.3 

197 

1 

8.0 

1810 

6 

4.0 

1810 

2 

0.3 

197 

7 

0.3 

388 

2 

0.3 

543 
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Model  evaluation 

To  evaluate  further  the  ef fectiveness  of  the  most  promising  of  the  uar.didute  models  - 
SNRT  -  vis-a-vis  the  Johnson  approach,  several  additional  sets  of  images  were  produced. 
Each  set  is  comprised  of  ship  images  at  three  different  ranges.  The  S.\’R  of  each  i.mage  is 
ad:usted  so  that  all  three  images  have  the  same  quality  as  defined  by  either  of  the  two 
models.  These  images  can  then  be  examined  side  by  sid"'  to  see  which  model  is  more 
effective  i.n  defining  quality.  Three  levels  of  task  difficulty  -  detection,  recogni  tier. , 
and  indenti f ication  -  were  treated  in  this  manner.  These  levels  were  established  by 
asking  observers  to  look  through  the  ordered  series  of  96  photographs  to  determine  the 
points  at  which  (a)  something  is  barely  detectable,  (b)  the  object  is  recognizable  as  a 
warship  rather  than  a  merchant  ship,  and  ic)  the  warship  is  identifiable  as  bei.ng  of  the 
KOTLIN  class.  Obviously,  since  only  KOTLI.V  images  were  used,  these  discrimination  levels 
were  defined  only  in  a  very  relative  sense,  and  are  not  meant  to  represent  an  absolute 
cr i ter ion , 


The  recognition  case  images  are  reproduced  in  figure  8  as  a  representative  e:<am,ple  o: 
tne  results  of  this  exercise.  Images  in  tne  upper  row  are  those  identified  by  the 
Jennson  model  as  being  of  equal  quality.  Imaqes  in  tht^  lower  row  are  of  equal  gua^ity  as 
defined  by  the  SNRT  model.  As  the  correlation  exercise  described  oreviousiy  suggests,  tht 
s:iRT  model  is  more  effective  in  trading  off  SNR  for  target  size.  Figure  9  shows  graph¬ 
ically  how  the  Johnson  model  overestimates  the  utility  of  low  SNR  imagery,  while  under¬ 
estimating  that  of  high  SNR. 


Ronfi*  -  22  Range  -  41  5  km  Rtmo,.  k  •- 

SNR  -  a  SNR  20  SNR  4 


Fig  8.  Images  of  equal  quality  as  assessed  by  the  Johnson  and  SNRT  models 


The  SNRT  model  was  also  more  successful  at  defining  consistent  detection  and  identi¬ 
fication  grade  imagery  than  was  the  Johnson  approach.  These  results,  and  the  results  of 
the  correlation  exercise  described  previously,  lead  us  "o  believe  that  the  SNRT  model  la 
more  accurate  in  predicting  image  utility  than  is  the  Johnson  model.  In  the  next  section 
the  practical  impact  of  this  conclusion  on  system  modeling  and  design  tradeoffs  will  be 
examined . 


Implications 


The  experimental  results  described  above  point  to  a  change  in  sensor  modeling  that 
will  impact:  (a)  the  prediction  of  field  performance  of  a  given  system  and  (b)  the 
tradeoffs  made  during  the  design  of  the  system. 
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Impact  on  performance  prediction 

To  explore  the  impact  of  the  use  of  SNRT  on  performance  modeling,  we  have  defined  a 
hypothetical  thermal  imaging  system  which  has  the  same  MTF  as  the  camera  used  in  our 
experiment  {N  =  185  TVL/PH) .  Assuming  the  task  is  recognition  as  defined  in  the  same 
sense  as  previously  described,  the  Johnson  and  SNRT  models  produce  the  two  SNR  versus 
range  curves  plotted  in  figure  9.  The  five  negatively  sloping  curves  describe  the  SNR 
generated  in  the  sensor  by  the  thermal  signature  of  a  warship  after  propagation  through 
a  slant  path  from  the  surface  to  the  sensor  altitude.  Range  is  normalized  by  the  sensor 
vertical  field  of  view.  The  extinction  coefficients  were  calculated  for  a  particular 
spectral  band,  sensor  altitude,  geographic  location,  and  sensor.  The  curves  represent 
conditions  of  SNR  that  would  be  exceeded  10,  20,  50,  80,  and  90%  of  the  time,  respec- 
tively.  The  zero  range  SNR  is  the  ratio  of  the  target  thermal  contrast  to  the  system 
noise  equivalent  thermal  contrast.  The  intersections  of  either  of  the  system  demand 
functions  with  the  target  strength  functions  occur  at  the  recognition  ranges  that  would 
be  exceeded  10,  20,  50,  80,  or  90%  of  the  time.  Under  the  10  percentile  condition,  the 
SNRT  model  predicts  a  recognition  range  35%  longer  than  the  Johnson  model.  rindAr  th^ 
more  typical  50  percentile  condition,  an  increase  of  26%  is  predicted  by  SNRT.  Repeating 
the  exercise  for  the  detection  case  produced  even  more  significant  differences  in  the 
predicted  ranges.  Under  median  weather  conditions,  the  SNRT  model  predicts  a  detection 

In  extremely  good  weather  the  discrepancy 


In  locations  at  lower  altitudes  where 
transmission  is  poorer,  the  SNRT  model 
predicts  ranges  shorter  than  those  generated 
by  the  Johnson  model.  Thus,  depending  on  the 
environment,  target,  task,  and  system,  the 
Johnson  model  can  produce  significant  range 
errors  in  either  direction. 

Impact  on  sensor  design 

The  Johnson  model  tends  to  underestimate 
the  degree  of  system  performance  increase  to 
be  gained  through  improvement  in  system  sensi 
tivity.  This  is  illustrated  in  figure  10, 
again  using  our  hypothetical  forward  looking 
infrared  (FLIR)  for  warship  recognition. 
Figure  10a  gives  system  demand  functions 
based  on  the  Johnson  model,  using  two  differ¬ 
ent  values  of  system  noise.  The  upper  curve 
represents  a  system  with  sensitivity  commen¬ 
surate  with  what  might  be  achieved  with 
current  detector  technology;  the  lower  curve 
represents  a  4:1  increase  in  sensitivity, 
such  as  might  be  achievable  using  more 
detectors.  The  target  strength  curves  are 
plotted  as  in  the  previous  example.  Note 
that  according  to  this  model  the  4:1  sensitiv 
ity  improvement  results  in  a  range  increase 
of  only  11%  for  the  50  percentile  condition. 
Figure  10b  shows  the  same  analysis  using  the  SNRT  model.  Here  the  range  increase  is  a 
more  substantial  25%.  As  was  the  case  in  the  previous  example,  the  discrepancy  between 
the  Johnson  model  and  SNRT  is  greater  for  the  detection  task,  and  less  for  the  identifica¬ 
tion  task.  The  relative  insensitivity  of  the  Johnson  model  to  SNR  improvement  unduly 
biases  design  tradeoffs  in  favor  of  resolution,  and  results  in  systems  which  are  less 
than  optimum  in  terms  of  effectiveness. 


range  43%  higher  than  the  Johnson  model, 
increases  to  over  60%. 


Fig  9.  Comparison  of 
performance  predictions 


Suninary 

Through  a  series  of  ship  images  produced  over  a  wide  range  of  SNR,  MTF,  and  range, 
we  have  shown  that  the  Johnson  criterion  tends  to  overestimate  the  utility  of  images  made 
at  low  SNR,  while  underestimating  the  quality  of  high  SNR  images.  This  results  in  range 
predictions  which  tend  to  be  pessimistic  in  situations  involving  strong  target  signatures, 
sensitive  sensors,  favorable  weather,  or  lower  order  tasks,  and  optimistic  in  situations 
at  the  other  extreme. 
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Fig  10.  Comparison  of  performance  predictions  for  current  generation  sensor  and 
increased  sensitivity  sensor  for  the  Johnson  and  SNRT  models 


We  have  developed  a  formulation  which  yields  a  higher  correlation  between  predicted 
image  quality  and  the  quality  as  assessed  by  a  group  of  observers  than  does  the  Johnson 
approach.  This  relatively  simple  model  drives  sensor  design  tradeoffs  more  in  the 
direction  of  sensitivity,  and  is  more  effective  in  accurately  predicting  effective  stand¬ 
off  range  over  a  broad  range  of  conditions. 
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